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Introduction - Visual Scanning Behavior 

The aircraft pilot has many sources of inforniftlion ir4)ut but the most important one 
curing instrument flight is probably the visual pathway. The pilot obtains 
information concerning aircraft state by cross-checking or scanning the flight 
insti'jments. T'.e exact method of scanning the instrument panel varies from pilot to 
pilot but thf;re are some basic features cernnon to a "good" scan pattern. Indeed, it was 
the early study by Fitts and his associates on instrument transitions which led to the 
familiar ’T' arrangenent of the mitjor flight instruments (Jones, et.al., 1946). 

A fundanental notion in the method discussed here is that a repetitive piloting task 
will invoke a regular visual scan (spatial /temporal pattern of eye movements) during 
instrument flight. If this notion is correct, then it may be postulated that 
external factors such as noise, interruptions, fatigue, etc. which interfeu vith tie 
piloting task may produce measurable changes ir. tie scanning behavior. Such a 
neasure vwulc be particularly attractive for quantifying workload since it would be both 
non-invasive ana objective. 

Qie of the main reasons for measuring nental 
workload is to predict situations under which 
piloting performance will fall below an acceptt±ile 
(safe) level. A persistent difficulty in wcrkload 
measurement appears to be the influence of skill 
on the performance vs. workload relationship as 
noted graphically in figure 1. Skilled pilots 
tend to maintain a high level of performance under 
increasing mental load until a precipitous 
performance decrement ceeurs during an overload 
situation. Cur work to date has suppx^rted this 
notion and led us to exp;lore the relationships 
suggested in figure 1. Space does not allow a 
oetailed discussion of this issue here; it is 
sufficient to note that the measures wo are using 
are dependent on subject skill (indeed the 
measures may eventually [)rovide good indicators of 
skill) and that novice pilots may be more 
desirable subjects if the effect of workload on 
pjerformiance is to be explored over wide ranges of task difficulty. 
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Vclhodology 

At present, our technique requires a piloting scenario v.tich forces a relatively 
sterectypx-d iristrument scar p-iittern and an indcp«ndent verbal (or possibly visual; see 
below) task of varying difficulty. The following is an abbreviated description of our 
previous experiments vliich should serve as a gerteral orientation to the nctlodology. 
Details of the mctlods, includir^; data analysis tc'clj'.iciies rrc discussed elsev.tiere (Tole, 
et al , 1982a & b). 

Pilot subjects are asked to maintain a general aviation flight simulator on a 
straight and level, ccristant sensitivity, Instrunent Landing System (ILS) course with a 
low level of turbulerec. /n ndditiorul pericdic verbal task wtose difficulty increases 
with frequency is used to increment the subject’s mental workload. Pilot lookpciiit on 
seven instruments (Attitude Indicator 'ATT, Directional Gyro TC\ Altimeter 
'ALT’, Vertical Speed Indicator 'VSl', Airspeed 'AS', Tbrn and Bank '*B\ and Glide 
Slopje /Localizer GSL') v/as measured using a Foneywcll oculonx'ter system (Middleton, 
etiil., 1977) mounted so as not to interrerc v.itli the subject's view of the instrur.cnt 
panel. This device is non-invesive and provides the time course of the pilot's 
instrument fixations to tlic nearest 1/30 sec. 

Tlie mental loading task was chosen so ns not to directly interfere with the 


(NAS A-Cfi-169^54) FaP STUDY Of PILOT N82-JC944 

UOHKLOAO. UUALlfICATION Cf hCBKLOAD VIA 
INSTBUflUNT SCAN (NASA) 6 p fcC A02/«f A01 

CSCL OSH Uncias 

G3/54 30402 


f IS 

OF PC. 0 'ALITY 

visual sianning of the pilot (i^. the task v.ould not rtt^aire the pilot to look away fron. 
the inSifunenls) while providiti0 corislent loading during the maneuver. TTic task 

used requires the pilots to respond to a scries of evenly spaced Ihret-i.irijer sequences 
(l.’ittentorn, presented audibly. The pilot was told that he trust respond to each 

sequence by indicating either "nlus” or "niinus" according to llie algoriUr.i : first 
nunl^r largest, second nurber smailest = "plus" (e^. 5-2-4), last nunber largett, first 
nuiTiber smal lest = "plus" (e.g. 1-2-3), otherwise, "minus" (e.g. 9-5-1). Performance 
\ts recorded b\ having the pilot press a 3-[osition rocker sv.itch mounted on the yx>ke 
uf for pius and •k)wn for minus. 

The mental workload experienced by the pilot is inversely prof>orl ional to the 
intervals between nunber sequences. T1 is relationship is given by tlie following arbitrary 
equation : 

(1) TD = 1/interval between task 

where TD is equal to imposed task di'ficulty. The four loading levels used in the 
current eiperimenls were intervals of continuous silence (i.e. no numbers presented), tin, 
five, and two seconc: v.hich hare corresponding tasl uifficulties of 0.0, OJ, 0.2, and 
C.5, respectively. Qalibration using a side task (Q)hrath, 1975) confirmed the relalivi 
(lifficulty of these nunter intervals. 

Several variables ere obtained from each of the two tasks in order to compute 
performajice scores. TTie scores developed ran between 0 percent ar.d 100 percent vnth 100 
[creent being obtained if the pilot never deviated fne tie intended path ir. space on 
tie ['iloting task, ard if ell nunber task stoutnccs were aiiswered correctly. 

Ferforniarce cii Ue piloting task was* est ir.vittd fric tie glide slop< and local ii:er 
errors. Ir ai'diilicr- to the usutT r.easure I'f total RdS error iron, tie inlendec flight 
p.nth, a measure of "smoothness" of ride, estimated from the frequency content of 
t'ji i 1 lotions about the intended pf:th wts also induced. It \ a:- iirbitrarily assumed 
tint a smooth rioe v.ould contain frequecies mostly less than 0.1 Hz. Under this 
assunptior, r.eesurement of the s[>ectral component of the aircraft dynandcs above 
C.l Hz. would indicate any decrenient in the ride quality. Combining' the ?c> of the 
spoctral power above 0.1 Hz for the glide slo[o and localizer with the FH'5 trrers for 
these instrunenls yields a performance score for the piloting task. This neasure rroy be 
combined with the score from the nunber task to obtaii j uul [ < i form.-ince score. 

In cider to assess the effects of skill on pjerforn.iux‘t and Rental vorklcac, an 
irdcperder.t laiwit itat ive neasure of skill was needed. A nodcl of pilot skill based on 
experience 'factors (c.g. total flight time, total time in type, yrs. since 
(crtificalion, etc.) was used for this purpose U'd lister, et al, 1973). 

Effects of Loading on the Visual Scan 

Instrunent dwell tine histccrairs la d the frequency of usage of different sequences of 
inslrunen.t fixiiticas itre both t.ffeclec' ty tU lofiding task. Pilots tend to stare at the 
prinery instrument Uuitude indicator in our experinents) as the level of difficulty ef 
the verbal task increases. Tlie percentage usage cf various fixiilicr. seeuences (eg. ATT- 
DD) also decieascd v.itk increasing task difficultv. [qth of these trends were nuch nore 
prevalent in novice subjects as compeared with skilled pilots. 

The piloting task in U»e basic experirrcnl is such Uiai tie pilot's scan can only lie 
on one of the 7 specified instruments although each fixation may be of arbitrary 
duration. Tlie time history of fixations has a form which is similar to that ol a 
conminicat ion sysiemi wliich can assume 7 discrete states with a vtr'irg cliiraticn in each 
state. Tlie orderliness of such a sy'sleni is related to the probtibil i t les with which it 
ccciip its IlJ cEfferint stoles. A system which always occupi^ the sane state or always 
Riidc the seme transitions Utvecn slates would thus quite orderly. In tlie case of 
instrur.ert sen, tlesc situations would tx [erallcled by storing and by a stcrectypied 
scijipalh respiecl i vely. 

Tliis concept of syslen. order may bt staled c*crpioclly using the maUiematical form for 
enlrcfiy frooi infornetion theory. The entropy of t scquiuci is defined as (Flijuior and 
Vicnvcr, 1949); 


D 

(2) Ho = - ripilog2Pi5 

1=1 

v.hcrc lb “ observed avenge entropy 

Pi = probabi 1 i t'- c f secuence i oocurrit^; 

D = Number cf different' sequences in lU- scfji 
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In the case of the insiiument scan, entropy has the units of bits/sequence and 
provides a treasure of tlic raiKlofiTiess (or orderliness) of the scanpath. Tlie higher the 
entropy, the nore disorder is [iiesent in the scan. The maximuni possible entropy is 

constrained by the expcriruntal conditions. The maxitTig". [ossible value, Fhiax. Hnax 
bc' calculated as fcl lota’s. In the most general cast, instririents mty be arranged in some 
nl.itrtirs fashion on the cockp>ii panel. For a given nurrber of instrinents, M, am sequence 
krgth N, the maxinuri number of different fujlion sequences is gi\en by: 

( 3 ) Q = M . = maximm number of sequences of length N 

The number of bits recjuired to uniquely encode all Q fcssible sequences is lcg2Q. The 
nag^iituce of this latter number also represents of the visual scan for the niarber 

cf instruments and sequence leigth beirg considercc. For example, with 7 instrunents tfie' 

value of C for secuences of " instruments is 56 which yields a corresponding Hnax~ 

In order To induce the effect of instrument dwell times in our measure, a term for 
entropy rate was defined as: 


(4) 


D 

Hrate =.^Hi/TTii 


\\tiere Fi ~ sequence 

dj = Average dwell tine for ith secuence 
D = Nun ber cf different fixi.tion sccuences 


While it is fcssiblc for pilots to ^lWJ^e rather ra{j id glances (\.ith dwell tines of 100 msec 
or less) at tfeir instrunents (lUirris and Q.risthi If , 1980) t fintici rate this 

high (10 f ixations/soc) rapidly leads to (Xrulorotcr fatigue. A rore realistic ivuage 
volte is probably about 2 f ixi.t ions /sec or less for a long period cf irstrurrent scan (say 
> 10 sec). Using this value (O.f sec/look) as the average di'.cll interval, the maxinum 
entropy rate for sequences of lengll 2 is calculated frcr.i equation 5 to be: 

^^Vate)max)" 5.8/0.5 * 2 f ixations/seq. = 6 bits/sec 


This number represents an upper 
bound. Since we suspect that the pilot 
niisl have sonc regularity ir Ms tr her 
scan, the numbers we would expect to 
(tuip iidcr actual fliglt ccrdit’''ns 
\ill [robably t>e lower. The observed 
average Hrete basic 

experiments was on the order of 1 
bit/sec. A tendency to stare under 
increased load should be reflecte^d by 
decreased entropy aid increased fixidion 
tires making Hpntp tend toward lower 
values under such (.-onditions. Figure 2 
plots HpQtf. vs number task difficulty 
for our test subjects. 

This data is nodelc-d by e.xpressirg 
Hrate exponential furiction of TT'. 

(7) Hrete = 0.2279 e-TP 

Thiis eouftion may be solved for task 
difficulty to yield 
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Fip.CJ.n! rop\ rule on lencih-I srcurncp* vs iroosfd 
Ifcsx ciM’.cuHv lor 6 piiCls trtliiivc s^ill levels 
sriov>n or tre rirr.t - r. ip: es t = lU-v i. 


(8) TD = -(0.06 + InaVflie)] 

which can bc used to predict the level of TD for a new subject under tlte conditions of the 
experinent reperted here. 

Autocorrelation 


The relationship tetween instrument scan frequency and nurber task presentation fucucncy 
provides valuable insight on hiow the task, aid thnefeii tti j i-i^c'oiatt<5 ncr.lal load, 
affects tl)c scanning piittern. fue to the periodic nature of the verbal task, the use or 
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UJKx'orrtlfilTtii ii> Miljic the sconniit bd.f:vici cic fcssible netlK>d for examining titis 
rchitionship. 

Ailocorrclnl ion was performed on the scanning data as follows. A sequence of 
instrument nunttrs versus tin<c was devcTcped from the da’a and stored on a disk. Due to 
tlic erbitrars nature of the assignnxrt of instrincnt nimbers, the autocorrclnt i(>i cf tl»e 
i ignel containing all instrUiiient nuntx'rs would iK>t iifccssti il^^ [rcdiice neaningful results. 
For this reasoi each cf tit stver instruTiuits were rxan.ined successively ty replacing tlic 
tin<' st-qucnce cf all instrunents with t sccojence lx(i)} where the value is ] fer tlie 
instrument being studied ut’ '■ foi ill cticr instriJT*i.ts. In order to elin irate tlic dc 
cxf.fcneni fer htu si«tral analysis, a zero-netx stxjuence {f(i)} was cenputed fron 
{x(i)l as fellows: 

fj(i) = xj(i) - Xj 

where xj(i) = 1 siet ified instrurrert j is being fixated and 0 othervnse 

= mean of {xj(i )) 

fhe sample autocorrelation of {fj(i)), or sample autocovariance of {xj(i)}, was 
calculated h\ the fornula: ‘‘ 

r. 

(10) Rj(k) = l/n.^fj(i) . fjU^ k) ] 

where F.jt* ' - i u'<<t rrclat icr secutrve fer instriri»ent j 

II = nunher of Sandies = total ru dun 1 ioi /tculcneier stnpling period (l/30th sec) 

'll is a.ttetrrclation was cor-tiUd for each of the sivtr instruritrls for each loading 
case on each pilot. In order to detect possible periodicity in the scan, the Fourier 
transforn cf *.li i ’..‘ueerre lat ion was JaUu '.ii [reduce the [C't.cr density s[iectrun.. Fron 
this a value for the dcr. inti t freciucncy may be obteintd. /j' example of tliis analysis is 
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r 1 C .3 JVuioi or re I Bl ions for pilot *4 (rrlrtivr skill Irvrl * usinp atiitudf indicfclcr 

ttotiro linrs incirilr lo-src irtcrvnls). Suirorr tasK ir.trrsbls ano associaica task 
cifficullics are a) no intervals - 0, D) lu sec - b.l, c) b sec - O.C, a) I sec - U.S. 

shown in Figure .3. Tliis shc\* tl( f ulteorrelat ions for rilet ■'*> (sr-oond highest skill 
level) for his attitude indicator c^i each cf tk four different r.cutal Ic'acir^; ciiscs. A 
change in the dominant frccjuency nay bc> seer, as tl-c lofidirg is increased. Tlie [lowcr- 
s(>ectral density calculations show the dominant frequencies for the low (10-sccond 
intervals), mcdiui (r<-st*cond intervals), wkI high {C-sc«cond intervals) levels of rental 


4 


Ur\iG - . ' i 

OF POOr? quality 

workload to be 0.0928 Hz, 0.1709 Hz, and 0.3175 Hz respectively. These frequencies 
correspond to periods of 10.78 seconds for the low, 5.84 seconds for the mediun, and 3J5 
seconds for the high level of mental workload. These periods are closely related to the 
rairber tasks period (11, 6, and 3 sec) given by the sun of the interval between nutter 
presentation and the time required to present the nutters. This implies, at least for this 
pilot, that the loading task directly influences the scan pattern. Wten no nutters are 
presented, the pilot scans his instrunents in a close-to-random manner and the density 
spectrun exhibits no dominant frequ^y. When the periodic task is applied, the scan 
becomes more and more periodic with increased task frequency. This demonstrates that the 
pilot has a tendency to multiplex the flying task and the number task for greater 
efficiency. Overload occurs when numbers are presented too rapidly for the pilot to 
efficiently multiplex both tasks. 

A similar behavior has been observed for all of the hi^ly skilled pilots who have 
participated in our experiments thus far. Novice pilots, however, do not seem to have any 
consistent pattern in their autocorrelation sequences. Most of these pilots showed little 
or no periodicity in their scans for any of the loading conditions. Che e^lanation may 
be that skilled pilots have a better developed ability to time multiplex several 
simultaneous tasks. 

Visual Seaming Measures Applied to the HP Flight Thsk I, Haiaially Floan 
ILS Approach and landing of TWo-Pilot Passenger Jet Transport 

We now briefly discuss the application of our techniques to the proposed study of 
workload during an ILS approach. TVro or Lhree factors mist be manipulated to use our 
techniques; l; a piloting task requiring a stereotyped scan path, 2) a verbally 
presented mental loadirg task, or 3) a visually present^ mental loading task. It is 
assumed that the cockpit to be used for the experiments may be outfitted with the NAS\ 
Langley oculometer system or an equivalent and that ample time will be allowed 
(approximately 5-10 minutes) for calibration of the oculometer before an experimental 
session begins. 

The proposed ILS approach scenario requires the use of a stereotyped scan path, 
though it should be enphasized that the task and hence the scan pattern is not constant 
throughout the scenario. (Recall that our earlier experiments forced a repetitive scan 
pattern to be developed over a long constant flight maneuver) Thus, the second to second 
level of loading due to the flight task and the corresponding instrunent scan will vary, 
albeit in a somewhat predictable fashion. We believe it is inportant to note that the 
difficulty of the ILS approach changes somewhat from instant to instant and almost 
certainly inplys higher workload as the runway threshold is neared. The additional verbal 
or visual loading task serves to 'hias" the total amount of nental load on the pilot with 
the goal of locating peaks in the load due to the piloting task alone. The notion here is 
that the workload due to the additional task is roughly additive with the instantaneous 
load due to the piloting task. The hope would be to bias the total load to a high enough 
level to demonstrate a performance decrement (which may be a non-linear function of 
loading) while at the same time hopefully observing a monotonic change in the measures of 
scanning behavior as a function of the increased load. 

Several levels of difficulty of the additional task are required. These may achieved 
in two ways. A constant level of difficulty may be inposed over the entire approach; this 
method is to be reconmended at present as we are not as wt sure how to analyze short 

segments of the scan f»ttern. Each level of difficulty of the iruiosed extra task would 

thus require a separate run. Since both the verbal and visual tasks are periodic, their 
respective difficulties m^ be altered during a run by changing the period between 
presentations of tlie task, 'mis method would seem more attractive if the piloting task 
were inoeed fixed over the entire run. 

The verbal task described above may be used as one means of biasing the loading 
level. This has been shown to work well in jut experiments and is easy to inplement and 
score. Its limitation is that it is not a task which would ordinarily be performed in the 
course of flight. It would be possible to modify the task to make it more like either a 
constant rate of radio communication or a rapidly updated manual computaMon of 
navigational coordinates. 

An alternate, visual version of this task is also possible and perhaps more 
appropriate for actual flight conditions. A small display could be mounted in a 

convenient point in the pilot's visual field. The display could present either a or a 

sign. At periodic intervals an auditory lieep" wuld signal that the pilot should 
observe this display and indicate (optionally) via a rocker switch whether the display is 
currently indicating ♦ or -. The interval between ’heeps" d.ierrrines the difficulty of 
this task and one possible measure of workload is the 5b of time the pilot is actually able 
to observe the display. 
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Eiltropy rate calculat iot’> ootid U' rede on the scanning data regardless; of v^hcthcr 
the visual or verbal loading task is used. Since both tasks are periodic, the 
autocorrelation t«liiii(^iit n^y also be applied. AlthoM^h we have not done it as yet, we 
expect that cross correlating the time of presentation of the imposed task with the 
scanning data is likely to yield good results esfiecially in the type of flight scep«irio 
proposed in this study. We expect tlmt a cl.aracteristic "signature" will appear in ttie 
crosscorrelation boUtcn the loading task and the instriment scan and that this signature 
will be altered via charges in task difficulty. 

Limitations and Pitfalls of the Itehnique 

There are a number of potential problems in applying our techniques. These are 
ennumerated below ; 

1. The piloting task being performed mist require instrurent scan. 

2. The scan mist be repetitive, at present, though we are working on methods (eg. cross 
correlation) for analyzing short se^ents of a scan pattern. 

3. An onboard oculcmeter required and mist be mounts in instrunent panel (NAS\ - Langley 
Research Cfenter has worked out many of the technical protdd.s l<vever). 

4. It may be necessary to calibrate without the pilot's cooperation due to time 
limitations in the proposed experiments. 

5. The behavior of the various rreasures of seen has not been examined under a wide variety 
of situations as yet. hence we are unable to comment on flight scenarios in which the tasK 
is Host applicable olkicr thian tte obvious requirement of some type of scanning betiavior. 

respite these ictentiil limitations, V( ne c-enfident enough about the mettKxlology 
that we believe it should ue included as one of the techniques (hopefully in conjunct. on 
with other) in the proposed study. 
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